A standard approach in pattern classification is to estimate thedistributions of the label classes, and then to apply the Bayes classifier tothe estimates of the distributions in order to classify unlabeled examples. Asone might expect, the better our estimates of the label class distributions,the better the resulting classifier will be. In this paper we make thisobservation precise by identifying risk bounds of a classifier in terms of thequality of the estimates of the label class distributions. We show how PAClearnability relates to estimates of the distributions that have a PACguarantee on their $L_1$ distance from the true distribution, and we bound theincrease in negative log likelihood risk in terms of PAC bounds on theKL-divergence. We give an inefficient but general-purpose smoothing method forconverting an estimated distribution that is good under the $L_1$ metric into adistribution that is good under the KL-divergence.
展开▼
